An electronic imaging system depends on an electronic image sensor to create an electronic representation of a visual image. Examples of such electronic image sensors include charge coupled device (CCD) image sensors and active pixel sensor (APS) devices (APS devices are often referred to as CMOS sensors because of the ability to fabricate them in a Complementary Metal Oxide Semiconductor process). Typically, these image sensors include a number of light sensitive pixels, often arranged in a regular pattern of rows and columns. For capturing color images, a pattern of filters called a color filter array (CFA), is typically fabricated on the pattern of pixels, with different filter materials being used to make individual pixels sensitive to only a portion of the visible light spectrum. The most common two-dimensional CFA pattern is the Bayer CFA pattern U.S. Pat. No. 3,971,065. When a CFA is employed, some way of combining spatially separated pixels that sample different spectral bands (different colors or possibly different wavelength bands outside the visible region) is required. This process is called demosaicing or CFA interpolation.
While the terminology used here started with color filter arrays, it is possible for a single channel image to have a color pattern like a CFA pattern, even though there is not a one to one correspondence between the pixels in the image and individual filtered photoreceptors on an image sensor. These images have spatially separated pixels that sample different spectral bands (different colors or possibly different wavelength bands extending outside the visible region). These images are referred to as color pattern images and include images captured with a color filter array as a subset of color pattern images.
A color pattern is defined by the combination of a minimal repeating unit and set of effective spectral sensitivities. Processing an image with an RGB Bayer color pattern to produce a smaller image with an RGB Bayer color pattern is known in the prior art, such as in U.S. Pat. No. 6,366,318. This processing changes the size of a color pattern image, but does not produce an image with a different color pattern.
For any of these color pattern images, if one or more of the wavelength bands extend outside the visible region, the resulting fully populated image may be a pseudo color image.
In this discussion, the full size of a color pattern image is defined as the pixel dimension of the single channel color pattern image, regardless of the color pattern. The result of color pattern interpolation is to produce an image with multiple color channels populated for each pixel. Often, the image output from color pattern interpolation has the same dimensions as the starting single channel color pattern image. Interpolation of a single channel color pattern image can also produce a smaller or larger output image, but in all cases, multiple color channels are populated for each pixel. It is common for the output image to be smaller than the input image, for example in preparing a low-resolution preview image from a large color pattern image.
A poor color pattern interpolation will produce color and spatial artifacts that are not consistent with the original scene. The Bayer pattern has a long history so there are a large number of techniques available for converting a Bayer color pattern image into a full color image. Because these techniques have been developed over an extended period of time they are fairly efficient and robust, providing good color pattern interpolation.
In development of image sensors with novel CFA patterns, there is a need for the development of complementary processing algorithms. At the same time, there is an existing base of well-understood algorithms and hardware optimized for existing color pattern patterns such as the Bayer CFA. There is a need to process images with novel color patterns using existing algorithms and hardware.